compare_models: using 2931 users to estimate model performance
PROGRESS: Evaluate model M0
PROGRESS: recommendations finished on 1000/2931 queries. users per second: 17001.6
PROGRESS: recommendations finished on 2000/2931 queries. users per second: 22365.4
Precision and recall summary statistics by cutoff
+--------+-----------------+------------------+
| cutoff | mean_precision | mean_recall |
+--------+-----------------+------------------+
| 1 | 0.0255885363357 | 0.00660202699609 |
| 2 | 0.0254179460935 | 0.0131865348344 |
| 3 | 0.024337541226 | 0.0195730015843 |
| 4 | 0.0223473217332 | 0.0237471406868 |
| 5 | 0.0208802456499 | 0.0279004544869 |
| 6 | 0.0195610144433 | 0.0313233195782 |
| 7 | 0.0186187064386 | 0.0347225124093 |
| 8 | 0.0175707949505 | 0.0380009590091 |
| 9 | 0.0167936616248 | 0.0408946127984 |
| 10 | 0.0159672466735 | 0.0431541003445 |
+--------+-----------------+------------------+
[10 rows x 3 columns]
PROGRESS: Evaluate model M1
PROGRESS: recommendations finished on 1000/2931 queries. users per second: 1282.29
PROGRESS: recommendations finished on 2000/2931 queries. users per second: 1277.46
Precision and recall summary statistics by cutoff
+--------+-----------------+-----------------+
| cutoff | mean_precision | mean_recall |
+--------+-----------------+-----------------+
| 1 | 0.185602183555 | 0.0580033551119 |
| 2 | 0.155237120437 | 0.0913773449342 |
| 3 | 0.13590355965 | 0.11735511541 |
| 4 | 0.122824974411 | 0.137502018055 |
| 5 | 0.111838962811 | 0.154550447846 |
| 6 | 0.103093369726 | 0.170952566142 |
| 7 | 0.096602817176 | 0.18653678385 |
| 8 | 0.0893039918117 | 0.196434939449 |
| 9 | 0.084082034952 | 0.206375680122 |
| 10 | 0.0797679972706 | 0.21799005233 |
+--------+-----------------+-----------------+
[10 rows x 3 columns]
Model compare metric: precision_recall